AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	1.06926	2.05
	2_DecisionTree	Decision Tree	logloss	0.59297	25.26
	3_Default_Xgboost	Xgboost	logloss	0.503254	51.29
	4_Default_NeuralNetwork	Neural Network	logloss	0.550027	12.91
	5_Default_RandomForest	Random Forest	logloss	0.547167	32.1
the best	Ensemble	Ensemble	logloss	0.502281	0.61

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: gini
• max_depth: 3
• num_class: 4
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

24.1 seconds

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0.625337	0.554326	0.846174	0.606154	0.756615	0.657998	0.741934	0.59297
recall	0.781145	0.63847	0.938822	0.318255	0.756615	0.669173	0.756615	0.59297
f1-score	0.694611	0.59343	0.890094	0.417373	0.756615	0.648877	0.737565	0.59297
support	297	863	3498	1238	0.756615	5896	5896	0.59297

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	232	61	4	0
Labeled as Major	116	551	86	110
Labeled as Minor	5	63	3284	146
Labeled as Moderate	18	319	507	394

Learning curves

Decision Tree

Tree #1

Rules

if (APR Severity of Illness Description > 1.5) and (Age Group <= 3.5) and (APR Severity of Illness Description <= 2.5) then class: Minor (proba: 98.38%) | based on 5,192 samples

if (APR Severity of Illness Description > 1.5) and (Age Group <= 3.5) and (APR Severity of Illness Description > 2.5) then class: Minor (proba: 79.81%) | based on 4,889 samples

if (APR Severity of Illness Description <= 1.5) and (APR Severity of Illness Code <= 3.5) and (Age Group > 2.5) then class: Major (proba: 55.22%) | based on 3,073 samples

if (APR Severity of Illness Description > 1.5) and (Age Group > 3.5) and (APR Severity of Illness Description > 2.5) then class: Moderate (proba: 60.9%) | based on 1,826 samples

if (APR Severity of Illness Description <= 1.5) and (APR Severity of Illness Code <= 3.5) and (Age Group <= 2.5) then class: Minor (proba: 45.15%) | based on 917 samples

if (APR Severity of Illness Description <= 1.5) and (APR Severity of Illness Code > 3.5) and (Patient Disposition > 6.0) then class: Extreme (proba: 55.25%) | based on 829 samples

if (APR Severity of Illness Description > 1.5) and (Age Group > 3.5) and (APR Severity of Illness Description <= 2.5) then class: Minor (proba: 65.56%) | based on 694 samples

if (APR Severity of Illness Description <= 1.5) and (APR Severity of Illness Code > 3.5) and (Patient Disposition <= 6.0) then class: Extreme (proba: 86.42%) | based on 265 samples

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence Extreme (Fold 1)

Dependence Major (Fold 1)

Dependence Minor (Fold 1)

Dependence Moderate (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• num_class: 4
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

1.4 seconds

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0	0	0.593284	0	0.593284	0.148321	0.351985	1.06926
recall	0	0	1	0	0.593284	0.25	0.593284	1.06926
f1-score	0	0	0.744731	0	0.593284	0.186183	0.441836	1.06926
support	297	863	3498	1238	0.593284	5896	5896	1.06926

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	0	0	297	0
Labeled as Major	0	0	863	0
Labeled as Minor	0	0	3498	0
Labeled as Moderate	0	0	1238	0

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of 4_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• num_class: 4
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

12.0 seconds

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0.720307	0.586359	0.894068	0.537099	0.76713	0.684458	0.765322	0.550027
recall	0.632997	0.617613	0.904803	0.51454	0.76713	0.667488	0.76713	0.550027
f1-score	0.673835	0.60158	0.899403	0.525578	0.76713	0.675099	0.765955	0.550027
support	297	863	3498	1238	0.76713	5896	5896	0.550027

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	188	102	1	6
Labeled as Major	60	533	23	247
Labeled as Minor	2	35	3165	296
Labeled as Moderate	11	239	351	637

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of 3_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: multi:softprob
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: mlogloss
• num_class: 4
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

50.1 seconds

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0.700704	0.611111	0.894384	0.574581	0.78036	0.695195	0.776015	0.503254
recall	0.670034	0.637312	0.915094	0.525848	0.78036	0.687072	0.78036	0.503254
f1-score	0.685026	0.623936	0.904621	0.549135	0.78036	0.69068	0.777833	0.503254
support	297	863	3498	1238	0.78036	5896	5896	0.503254

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	199	89	3	6
Labeled as Major	73	550	28	212
Labeled as Minor	4	29	3201	264
Labeled as Moderate	8	232	347	651

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence Extreme (Fold 1)

Dependence Major (Fold 1)

Dependence Minor (Fold 1)

Dependence Moderate (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of 5_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: logloss
• num_class: 4
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

30.9 seconds

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0.644258	0.673267	0.874562	0.534258	0.769166	0.681586	0.762042	0.547167
recall	0.774411	0.472769	0.926815	0.529079	0.769166	0.675769	0.769166	0.547167
f1-score	0.703364	0.55548	0.899931	0.531656	0.769166	0.672608	0.762284	0.547167
support	297	863	3498	1238	0.769166	5896	5896	0.547167

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	230	51	2	14
Labeled as Major	105	408	39	311
Labeled as Minor	5	5	3242	246
Labeled as Moderate	17	142	424	655

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence Extreme (Fold 1)

Dependence Major (Fold 1)

Dependence Minor (Fold 1)

Dependence Moderate (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Default_Xgboost	4
4_Default_NeuralNetwork	1

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0.705263	0.611479	0.893344	0.574506	0.780699	0.696148	0.775666	0.502281
recall	0.676768	0.641947	0.917095	0.516963	0.780699	0.688193	0.780699	0.502281
f1-score	0.690722	0.626343	0.905064	0.544218	0.780699	0.691587	0.777703	0.502281
support	297	863	3498	1238	0.780699	5896	5896	0.502281

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	201	89	2	5
Labeled as Major	73	554	25	211
Labeled as Minor	4	28	3208	258
Labeled as Moderate	7	235	356	640

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back